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A STUDY OF ERRORS IN TESTS OF ADDING ABILITY 1 



C. L. PHELPS 
Leland Stanford Junior University 



In some of the recent tests of adding ability the results have been 
based almost entirely on the rapidity with which combinations were 
attempted and not much attention has been given to the errors 
that were made in taking the tests. Justification of this failure 
to make a careful study of errors is found in the claim that speed 
and accuracy show such a close positive correlation in these tests 
that checking for errors can safely be neglected. If this claim is 
true, the only remaining reason for making an analysis of errors 
is for diagnostic purposes. But this reason alone seemed sufficient 
to justify the writer in making an examination of a large amount 
of data in order to find out how often errors are made on the various 
combinations, what per cent of a group make them, how the com- 
binations rank in difficulty, and what kind of mistakes are made 
on them. 

SOURCE AND DESCRIPTION OF DATA 

The material upon which this study is based was collected by 
Otis and Davidson 2 for their investigation of the reliability of 
standard scores as a measure of adding ability. It was collected 
from the eighth grade of eight of the larger grammar schools of 
San Jose, California, a city of about 50,000 population. It will 
be seen by reference to their study that their tests consisted of 
25 arrangements of the combinations varying slightly in order from 
the Courtis Test No. 1, that the tests were one minute in length, 
that five were given each day for five days, that they were all given 
by the same person in exactly the same way, and that 270 children 
were thus examined. 

1 This study was suggested and guided by Professor Percy E. Davidson of Leland 
Stanford Junior University. 

2 "The Reliability of Standard Scores in Adding Ability," Elementary School 
Teacher, October, 1912. 
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Of this material the papers of 238 children have been checked 
for errors and the results studied. Of the 32 sets of papers rejected 
for this study 20 were five tests short of the full number and twelve 
were eliminated because they were not representative and would 
have vitiated the results if they had been included. For example, 
when it was found that the lower figure had been written for the 
total of several combinations in succession, it indicated that no 
attempt had been made to add and the sets of papers containing 
this error were left out. This type of error, with others which 
could not in fairness be included, resulted in the elimination of the 
twelve sets referred to as not representative. The 238 sets remain- 
ing, consisting of 5,950 tests, are considered entirely fair for the 
purposes of this study. 

METHOD OF MARKING ERRORS AND TABULATING RESULTS 

In checking the errors each paper was examined carefully and 
all mistakes marked. At first a separate marking was made of 
combinations which had been missed at the first stroke of the pencil 
and corrected by a second stroke. But this was later abandoned 
and only those errors were counted which had not been corrected 
at all. 

When the errors were all marked, a system of tabulation was 
arranged in such a way that the errors on each combination could 
easily be totaled for each pupil as well as for the whole group. 
This was done for each school separately, and then the results 
were collected into one table which is here presented as Table I. 

DISPLAY AND DISCUSSION OF TABLES 

This table shows gross results just as they appeared when the 
first tabulation was completed. It is reproduced here in order to 
show how the errors were distributed among the eight schools and 
on the various combinations. On examination several things can be 
seen in regard to this point. One is that there is a rather erratic 
distribution of errors in the ones and twos; another is that there 
is a more regular distribution in the other combinations; a third 
is that the number of errors increases with the increase in size of the 
combinations ; and a final one is that the errors on the doubles are 
almost as erratic in their distribution as on the ones and twos. 
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Table I takes no account, however, of the fact that in any 
arrangement of combinations for tests having a time limit certain 

TABLE I 

Showing Errors on Each Combination for Each School and for the Whole 

Group 



Combinations 



0+0 
+ 1 

+ 2 

O + 4 

°+5 

0+6 
o+7 
0+8 

+ Q 

I+I 
1+2 

i+3 
1+4 
i+5 
1+6 

1 + 7 
1+8 

1 + 9 

2 + 2 
2 +3 
2+4 
2+5 
2+6 

2 + 7 
2+8 

2+9 

3+3 

3+4 
3+5 
3+6 
3+7 
3+8 
3+9 

4+4 
4+5 
4+6 
4+7 
4+8. 
4+9^ 



School 
No. 1 



40 

Pupils 



10 
22 

21 

6 
9 



6 

19 

7 
8 

3 

7 

15 

5 
10 
12 

8 
23 



3 

2 

21 
3° 
32 



School 

No. 2 



Pupils 



1 
1 

3 

1 
o 

4 

4 

2 

8 

16 

T 5 

17 

6 

6 

T 

7 
ti 

9 

2 

T 7 
7 
6 

4 
1 
8 
IS 
30 
34 
46 



3 
12 

52 
23 
56 



School 
No. 3 



14 
Pupils 



2 
2 

14 

7 
11 

9 

16 

o 
2 
2 

12 
7 

23 



School 
No. 4 



23 
Pupils 



24 

IT 

i6 
IQ 

4 
4 

1 
4 
10 
8 
3 

IT 

5 
7 

15 
1 

11 
6 

11 
21 
19 

o 
3 

7 

8 

14 

t9 



School 
No. 5 



33 
Pupils 



19 
5 
6 

5 
6 

9 
i 

5 
5 
4 



6 
21 
12 

3 
17 

4 

3 

5 

20 

10 
T 9 



5 
7 
9 
4 
7 
30 
39 

3 

4 
6 



School 
No. 6 



48 
Pupils 



1 
6 
1 
4 
o 

4 

12 

4 



5 
18 

43 
16 
16 
19 

9 



4 
3 

8 

3 
32 
11 

!3 

12 

13 
12 
11 
36 

44 
45 



9 

4 

44 

21 

19 



School 

No. 7 



Pupils 



5 
4 
8 

12 

7 

21 

16 

5 

9 

o 

5 
7 
9 

IT 

2 2 

6 
15 

19 



12 

11 
59 
52 



12 

4 

50 

19 

22 



School 
No. 8 



40 
Pupils 



7 

30 

12 

17 

18 

2 

6 

3 
1 
10 
18 
3 
41 
ti 

13 

15 
3 
6 

4 

6 
11 
39 

o 

7 

4 

37 

17 

TO 



Total 



23 

J5 
16 

I? 
14 
3i 

3 
14 
37 

9 

16 

38 

53 

T 54 

86 

132 

144 

37 

65 



32 

59 

102 

42 

158 

52 

7i 

87 

40 

78 

7i 

120 

231 

3°4 



43 

4i 

232 

142 

192 
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TABLE I— Continued 



Combinations 



School 
No. i 



40 
Pupils 



School 
No. 2 



Pupils 



School 
No. 3 



14 
Pupils 



School 
No. 4 



23 

Pupils 



School 
No. 5 



33 

Pupils 



School 
No. 6 



48 
Pupils 



School 

No. 7 



28 
Pupils 



School 
No. 8 



40 
Pupils 



Total 



5+5- •■ 
5+6... 
5 + 7- •• 
5+8... 
5+9- ■■ 

6+6. . . 

6+7... 
6+8. .. 
6+9... 

7 + 7- • • 
7+8... 
7+9- • • 

8+8... 
8+9... 

9+9. . . 

Total 



17 
16 

78 

2 
81 
14 
3 2 

2 

35 

7i 

14 
14 



783 



23 
35 
63 
41 

1 
11 
15 



3 

11 

75 

n 
27 



851 



1 

5 

6 

26 

21 

1 

10 

o 

7 

2 

16 
5 



1 

10 
11 
Si 
47 

3 

13 

1 

35 

1 

6 

38 

20 
13 



295 



542 



15 
16 

24 
19 



14 
6 

23 



5 
9 

61 

6 
31 

3 
619 



o 
18 

J 3 

44 
3° 

7 
24 

3 
50 

1 

18 
64 



21 
3 



848 



26 
58 
78 

1 
12 

18 
42 

2 

19 
40 

25 
20 



826 



O 

2 5 
II 

2 5 
30 



5 
33 

o 
23 
41 

12 
16 



624 



4 
135 
134 
3f>9 



21 
1 86 

62 
320 

16 

137 
395 

118 
145 



combinations will be attempted oftener than others because some 
pupils will work more rapidly than others. For this reason only 
approximate results can be obtained for those combinations which, 
in the arrangement of the tests, fall beyond the average number of 
attempts made by the pupils tested. Accurate results can be 
obtained, however, for all those combinations which fall safely 
within this limit. These will all receive practically an equal 
number of trials because of the slight variation in the order of the 
combinations for the different tests and because the average devia- 
tion from the central tendency with regard to the number of com- 
binations is not high, except in one school where it amounts to 
10.6 combinations. But in this case the average for the school 
is so far above the average for the whole group that results are not 
affected. In the other cases they would be affected but slightly, 
For combinations which do not fall within the range of average 
number of attempts, a certain amount of data is available and 
fairly accurate conclusions might be drawn concerning some of 
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them. But it is perhaps safer to eliminate them from this considera- 
tion. Combinations affected in this way are: o-f-o; i+i; 4+4; 
4+9i 5+ 5' an d 6+8. There are also certain combinations which 
are repeated in inverse order within the limits noted. These have 
all been credited with twice as many trials as the others. 

These explanations prepare the reader for Table II, which is 
derived from Table I. In this is shown the percentage of error 
made on each combination, the basis for calculation being the 
actual number of times it was attempted. This table is considered 
as fair an index to the relative difficulty of the combinations as 
can be derived by this method from the data at hand. 

TABLE II 

Showing Percentage of Errors Based on the Number of Trials for Each 

Combination 



Combinations 


Errors in 
Percentage 


Combinations 


Errors in 
Percentage 


Combinations 


Errors in 
Percentage 


O+O 

O+I 


•39 
•25 
•14 
.28 
.24 
■52 
•°5 
•24 
.62 

•15 

•27 
.64 
•45 

1.30 
.72 

2.22 

2.42 
.62 

1 .09 


2 + 2 

2+3 

2+4 


■37 
■54 
■5° 
.86 

•7i 
i-33 

.88 
1. 19 
1.46 

.67 
131 
1. 19 
2.02 
1.94 

2-55 
. 12 

•72 

.69 

i-95 


4+8 

4+9 

5+5 


2.38 


+ 2 


.07 

2. 27 




2 + 5 

2 + 6 


5+6 

5+7 


0+4 


o+s 




5+8 

5+9 


3-IO 
2.50 

•35 
1-56 

I .04 


0+6 


2+8 

2+9 

3+3 

3+4 

3+5 

3+6 . 


0+7 


6+6 

6+7 

6+8 

6+9 

7 + 7 

7+8 


0+8 




i + i 


1+2 


■ 14 

2.30 
3-32 
1.98 
2-44 
-3° 


i+3 

1+4 


3+7 

3+8 

3+9 

4+4 

4+5 

4+6 

4+7 


1 + 5 


8+8 

8+9 

9+9 


1+6 

1 + 7 


1+8 


1+9 













* Unsatisfactory data. 

But it may be held by some that the method used here is not a 
fair one because it is based entirely on the number of errors made 
and does not take into account the number of children making the 
errors. It may further be held that, because of the large number 
of trials possible, a very few repeaters could make the entire num- 
ber of mistakes credited to any combination. When such a pos- 
sibility suggested itself all the papers were checked again to see 
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exactly what the facts were concerning this possibility. The find- 
ings in this search were tabulated by schools so as to show how 
many children missed each combination a single time and how 
many missed it two or more times. This table is not reproduced 
here, on account of lack of space, but two condensed ones, show- 
ing the rank of 20 of the combinations above the ones, are taken 
from it along with a similar table taken from Table II. In these 
three tables 20 combinations are ranked in difficulty in accord with 
(1) the number of times they were missed, (2) the number of chil- 
dren that missed them, and (3) the number of children that missed 
them two or more times. These tables are presented, for purposes 
of comparison, as Tables IIL4, 1I1B, and IIIC, respectively. 



TABLE IIL4 



TABLE IILB 



TABLE IIIC 







Errors 






No. of 


Percent- 






No. of 
Chil- 


Percent- 
age of 
Chil- 
dren 

Making 


Rank 


Combi- 
nations 


in 
Percent- 
age 

of At-„ 
tempts 


Ra 


, Combi- 
nK nations 


Chil- 
dren 
Making 


age of 
Children 
Making 


Rank 


Combi- 
nations 


dren 

Making 

2 or 










Errors 


Errors 






More 
Errors 


More 
Errors 


I . . . . 


9 + 7 


3-32 


I 


■•• 9+7 


120 


50.42 


I. 




9+7 


63 


26.47 


2. 




8+5 


3.10 


2 


• ■ ■ 9+3 


109 


45.80 


2. 




9+3 


59 


24.79 


3- 




9+6 


2.69 


3 


... 9+6 


94 


39-49 


3- 




9+5 


49 


20.59 


4- 




9+3 


2-55 


4 


... 8 + 5 


91 


38.23 


4- 




8+5 


47 


19-75 


5. 




9+S 


2.50 


5 


■•■ 9 + 5 


86 


36.13 


5- 




9+6 


38 


15.96 


6. 




9+8 


2.44 


6 


... 7+6 


79 


33 • 19 


6. 




7+4 


37 


iS-55 


7- 




8+4 


2.38 


6 


• ■ • 7+2 


79 


33 19 


7- 




9+4 


33 


13.86 


8. 




8+7 


2.30 


7 


... 8+3 


74 


31.09 


8. 




8+3 


32 


13-44 


Cl- 




6+5 


2. 27 


8 


... 7+4 


72 


30-25 


9- 




7 + 2 


31 


13.02 


io. 




7+5 


2.25 


9 


... 8+8 


7i 


29.83 


9- 




8+7 


31 


13.02 


11 . 




7+3 


2.02 


10 


... 9+8 


67 


28.15 


10. 




9+8 


26 


10.92 


12 . 




8+8 


1.98 


10 


... 8+7 


67 


28.15 


11. 




7+6 


25 


10.50 


13- 




7+4 


i-9S 


11 


• • • 9+4 


60 


25.21 


11. 




6+5 


25 


10 . 50 


14- 




8+3 


1.94 


12 


... 5 + 2 


59 


24.79 


12. 




8+8 


24 


10.08 


IS- 




7+6 


1.56 


13 


... 6+s 


58 


24-37 


13- 




7+3 


23 


9.66 


16. 




3+3 


1 .46 


14 


... 8+4 


52 


21.85 


14- 




8+4 


21 


8.82 


17- 




7 + 2 


i-33 


14 


... 9+2 


52 


21.85 


14- 




5 + 2 


21 


8.82 


18. 




5+3 


131 


15 


•■■ 7+5 


48 


20.17 


IS- 




7+5 


20 


8.40 


19- 




6+3 


1. 19 


15 


••• 7+3 


48 


20. 17 


16. 




9+2 


17 


7-i4 


19. 




9+2 


1. 19 


16 


... 6+3 


44 


18.48 


17- 




8+6 


15 


6.30 



From these tables it can be seen that there is a close correspond- 
ence in results, regardless of whether they are based on the number 
of times the combination was missed, on the number of children 
who missed it, or on the number who missed it two or more times. 
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It can also be seen that the higher combinations make the greatest 
showing of difficulty from all three points of consideration. The 
totals for the number of mistakes, the number of children who 
made them, and the number who made them repeatedly are all 
highest for the higher combinations. 

The fact that the tests were repeated 25 times is significant 
in this connection. We may assume that any child might miss a 
combination once in 25 times on account of a slip of the pen or 
for some reason other than lack of knowledge of the combination, 
but it is not to be supposed that this would be done repeatedly. 
It may not be of the highest importance to note that 120 children 
out of 238 missed the combination 9+7, but it certainly is signifi- 
cant to know that 63 of them missed it two or more times. If an 
arbitrary standard were to be set up and it should be agreed that 
two or more errors in 25 chances on any combination should be 
taken to mean uncertainty concerning that combination, then it 
could be stated that from 6 per cent to 26 per cent of the children 
taking these tests were uncertain about some one or more of the 
twenty combinations included in the tables above. This would 
indicate that there are many children who make mistakes on the 
higher combinations because of lack of knowledge of them or uncer- 
tainty concerning them. 

The errors on some of the combinations, especially on the ones 
and the doubles, seem to be out of proportion to the general indi- 
cations of their difficulty, but this is because of the existence of 
certain type errors. Table IV shows the principal errors of this 
kind found in this study and indicates the explanation of the 
erratic showing of errors on certain of the combinations previously 
referred to. 

One form of these errors is multiplication which is found mainly 
in the ones, though it is also found in certain other combinations. 
The 3+3 combination is a striking example of this. In three of 
the eight schools tested no other kind of mistake was made on it, 
while only 8 out of a total of 87 errors were due to any other cause. 
This kind of type error is not found in any other double, unless it 
is in the 2+2 where it makes no difference in the result. 

In 8+8=18 a different form of type error is found which 
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accounts for nearly all of the mistakes made on the 8+8 combina- 
tion. For example in School No. 7, 16 out of 28 children made this 
error one or more times. 



TABLE IV 

Showing Principal Type Errors As Indicated by the Number of Times the 
Same Mistake Was Made in Each Case 



Combinations 

1+4=6 

1+4 = 4 

Miscellaneous . 

1+6 = 6 

1+6 = 8 

Miscellaneous . 

i+7 = 9 

i + 7 = 7 

Miscellaneous . 

2+7 = 7 

2+7 = 8 

2+7 = 11 

Miscellaneous. 

3+3 = 9 

Miscellaneous . 

3+9 = ii 

3+9 = i3 

3+9 = 9 

Miscellaneous. 

8+8 = 18 

Miscellaneous . 



School 


School 


School 


School 


School 


School 


School 


School 


No. 1 


No. 2 


No. 3 


No. 4 


No. s 


No. 6 


No. 7 


No. 8 


5 


2 


I 


18 


17 


26 


7 


24 


3 


5 


3 


3 


3 


14 


5 


4 





1 


3 


3 


2 


3 





2 


19 


9 


4 


11 


11 


11 


17 


II 


3 


4 





3 


7 


3 


3 


2 





2 





2 


3 


1 


1 


4 


11 


iS 


21 


17 


8 


13 


9 


12 


8 


2 


I 


1 


3 


5 


4 


4 


2 


O 


O 


1 


1 


1 


3 


2 


4 


s 


5 


6 


9 


14 


13 


3 1 


1 


II 


3 


4 


2 


8 


6 


3 














3 


4 


2 


6 


3 


I 





1 


5 


6 


1 


1 


14 


3 


2 


14 


S 


9 


17 


IS 


1 


1 





1 





3 


2 





34 


24 


13 


12 


25 


35 


4i 


27 


2 


18 





4 


5 


4 


3 


2 


10 


2 


3 


2 


8 


3 


5 


9 


2 


2 





1 


1 


3 


3 


1 


13 


9 


8 


17 


5 


20 


22 


10 


1 


2 





3 


1 


2 


3 


2 



Total 



100 
40 
14 

93 

25 
13 

106 
28 
10 

87 
38 

15 
18 

79 



211 

38 
42 
13 

104 
14 



The 9+3 combination also presents peculiar results. Theoreti- 
cally, 9+3 should equal 13 as often as n. Practically, it does not. 
In every school n is strongly the favorite error, while the total 
shows more than 5 to 1 in its favor. 

Another point of interest is shown by Table V. It can be 
seen from this that there is a considerable difference in the number 
of errors made by the different groups, the average number per child 
varying from 15.60 in School No. 8 to 70.92 in School No. 2. It 
seems significant that there should be such a wide range, but the 
real importance of the table becomes apparent when it is noticed 
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that the school which had the smallest number of errors per pupil 
had the slowest speed record, and that the school which had the 
largest number of errors had the highest record for speed. 

TABLE V 

Showing Average Number of Errors per Pupil, Average Number of Combi- 
nations Attempted, and Rank of the Schools accord- 
ing to Speed and Accuracy 



Schools 



Average Number of 


Errors per Pupil 


19-57 


70 


92 


21 


07 


23 


56 


18 


76 


17 


67 


29 


SO 


15 


60 



Rank Based 
on Accuracy 



Average Number of 

Combinations 

per Pupil 



Rank Based 
on Speed 



I 
2 

3 

4 
5 
6 

7 



69 
76 

7i 
70 

73 
62 
68 
59 



77 
08 
78 
13 
75 
04 

5° 
82 



From this table it is apparent that if the San Jose schools were 
ranked according to the number of combinations attempted in 
these tests and according to the errors made in the tests, the order 
of the schools would be almost reversed. 

SUMMARY AND DISCUSSION OF RESULTS 

In the introduction of this paper it was stated that this study 
was undertaken to find out how often errors are made on the com- 
binations, what percentage of the group make them, how the com- 
binations rank in difficulty, and what kind of mistakes are made 
on them. Something has been learned about all of these questions 
and the results have been presented in the foregoing tables and 
discussions. They may be summarized as follows: 

1. Except in the zero combinations, where the errors were few, 
and in the ones where they were erratic because of the influence 
of type errors, a fairly regular increase was found in the number of 
errors made and in the number of children making them as the com- 
binations increased in size. This finding indicates either that not 
enough attention has been given to the teaching of the higher 
combinations or that their inherent difficulty is greater than has 
been supposed. The fact in the case could be definitely determined 



.38 THE ELEMENTARY SCHOOL TEACHER 

by experimentation, to the very great advantage of practice in 
teaching the combinations. 

2. It was found that not only was there an increase in mistakes 
and the number of children making them as the combinations 
increased in size but that a large percentage of the children made 
mistakes on each of the larger combinations. On certain combina- 
tions from a third to a half made mistakes, while about half of this 
number made two or more mistakes on the same combination. 

3. The rank in difficulty of the higher combinations is fairly 
well shown. For example, 9+7 was missed 3.32 per cent of the 
times it was attempted and 50.42 per cent of the children missed 
it. From this it can be seen that half of the children must have 
missed it, on the average, twice the percentage of the times credited 
to all. It can therefore be said that half the children taking these 
tests missed this combination about 6f per cent of the times they 
attempted it. Practically the same can be said of 8+5. A third 
or more missed 9+6, 9+3, and 9+5 an average of at least 75 per 
cent of their trials. And a fourth or more missed 9+8, 8+4, 8+7, 
6+5, 7+5, 7+3, 8+8, and 7+4 an average of more than 7 per 
cent of their attempts. These results are derived from Tables 
1114 and IILB and, besides giving an idea of their relative diffi- 
culty, demonstrate rather clearly the lack of knowledge of the 
higher combinations. 

4. It was found that there were certain kinds of errors which 
were peculiar to certain combinations throughout all the schools tested. 
These were called type errors. They were found to occur on several 
combinations and to account for a large part of the errors made 
on those combinations. Pedagogically, this finding is important, 
for, while the mere knowledge that a certain error is a type error 
may not suggest to the teacher the best method of dealing with it, 
it ought to prevent the ignoring of it as a slip of the pen or an 
accident. 

In conclusion, the fact must be noted that some of the results 
found in this study are not in harmony with certain statements 
concerning tests of this kind that are widely known and relied 
upon. Reference is made here to two statements made by Courtis 
in his "Instructions for Making and Recording Scores" in giving 
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the Courtis Standard Tests. The first statement, which applies 
to four of eight of the Courtis Tests, is that " Ignorance is shown, not 
by incorrect answers, but by reduced speed." That is, he holds 
that speed and accuracy in such tests show a close positive correla- 
tion. In this study no such correlation was found. The slowest 
group did the most accurate work and the fastest group made the 
most errors. The second statement is that "Mistakes, except 
for the zero combinations, are very few." In this study, mistakes 
were found to be fewest in the zero combinations and to gradually 
increase in number as the combinations increased in size, with certain 
exceptions which have been noted. And, finally, the results of this 
study do not justify the ranking of groups by scoring their speed 
record without taking into account the errors made on the tests 
taken. On the other hand, they indicate that speed records alone 
may be unfair and sometimes directly opposed, as they were in some 
cases in this study, to the findings of a tabulation of errors. 



